Method and system for providing complementary information for a video program

ABSTRACT

A method for providing complementary information  226  for a video program is provided that includes receiving complementary information  226  for a video program. A query is received from a consumer. The query is related to a specified portion of the complementary information  226 . A query response is provided to the consumer based on the specified portion of the complementary information  226.

This application claims the benefit of U.S. Provisional Application No.60/369,358, filed Apr. 2, 2002.

The present invention relates generally to video content augmentationand, more particularly, to a method and system for providingcomplementary information for a video program.

When viewing video programs, consumers may desire to receive informationrelated to the video program, such as actors' names. Currently availabletechniques for providing this type of information include audiocommentary, typically from the director of a movie, available on a DVD.In addition, electronic program guides available over cable andsatellite may offer limited credits for a particular program, such asactors' names and a date associated with the program. Another techniquefor providing additional information to the consumer includes enhancedtelevision broadcasts that allow content creators to insert theinformation into real-time broadcasts.

However, electronic program guides are not synchronized to the program,while enhanced broadcasts do not support recorded material. In addition,DVDs and enhanced broadcasts do not allow consumers to have control,with the possible exception of activating or deactivating thesefeatures. DVDs also cannot update the information provided, making someof the information possibly obsolete or inaccurate at a later time.

In accordance with the present invention, a method and system forproviding complementary information for a video program are providedthat substantially eliminate or reduce disadvantages and problemsassociated with conventional systems and methods. In particular,complementary information is provided to a consumer when thatinformation is requested by the consumer, complementary information maybe synchronized with a video program, control over access and navigationof the complementary information is provided to the consumer, and thecomplementary information may be updated.

According to one embodiment of the present invention, a method forproviding complementary information for a video program is provided. Themethod includes receiving complementary information for a video program.A query is received from a consumer. The query is related to a specifiedportion of the complementary information. A query response is providedto the consumer based on the specified portion of the complementaryinformation.

According to another embodiment of the present invention, a system forproviding complementary information for a video program is provided thatincludes a memory and a processor. The memory is operable to store userdata tables and complementary information. The processor is operable toreceive complementary information for a video program, to receive aquery from a consumer, the query related to a specified portion of thecomplementary information, and to provide a query response to theconsumer based on the specified portion of the complementaryinformation.

Technical advantages of one or more embodiments of the present inventioninclude providing an improved method for providing complementaryinformation for a video program. In a particular embodiment,complementary information is provided to a consumer when thatinformation is requested by the consumer. In addition, information thatmay be requested by the consumer may be anticipated and stored, alongwith user data that is specific to the consumer. The consumer-specificdata may then be used to provide personalized complementary information.

Other technical advantages of one or more embodiments of the presentinvention include synchronizing complementary information with a videoprogram, providing control over access and navigation of thecomplementary information to the consumer, and updating thecomplementary information.

Other technical advantages will be readily apparent to one skilled inthe art from the following figures, description, and claims.

Before undertaking the DETAILED DESCRIPTION, it may be advantageous toset forth definitions of certain words and phrases used throughout thispatent document: the terms “include” and “comprise,” as well asderivatives thereof, mean inclusion without limitation; the term “or,”is inclusive, meaning and/or; the phrases “associated with” and“associated therewith,” as well as derivatives thereof, may mean toinclude, be included within, interconnect with, contain, be containedwithin, connect to or with, couple to or with, be communicable with,cooperate with, interleave, juxtapose, be proximate to, be bound to orwith, have, have a property of, or the like; and the term “controller”means any device, system or part thereof that controls at least oneoperation, such a device may be implemented in hardware, firmware orsoftware, or some combination of at least two of the same. It should benoted that the functionality associated with any particular controllermay be centralized or distributed, whether locally or remotely. Inparticular, a controller may comprise one or more data processors, andassociated input/output devices and memory, that execute one or moreapplication programs and/or an operating system program. Definitions forcertain words and phrases are provided throughout this patent document,those of ordinary skill in the art should understand that in many, ifnot most instances, such definitions apply to prior, as well as futureuses of such defined words and phrases.

For a more complete understanding of the present invention and itsadvantages, reference is now made to the following description taken inconjunction with the accompanying drawings, wherein like referencenumerals represent like parts, in which:

FIG. 1 is a block diagram illustrating a video display system operableto provide personalized news to a consumer in accordance with oneembodiment of the present invention;

FIG. 2 is a block diagram illustrating the content augmentationapplication of FIG. 1 in accordance with one embodiment of the presentinvention;

FIGS. 3A-F illustrate the user profile tables of FIG. 2 in accordancewith one embodiment of the present invention;

FIGS. 4A-C illustrate the profile data matching tables of FIG. 2 inaccordance with one embodiment of the present invention;

FIG. 5 illustrates the information content matching table of FIG. 2 inaccordance with one embodiment of the present invention;

FIG. 6 is a block diagram illustrating a summary screen that may bedisplayed by the video display system of FIG. 1 in accordance with oneembodiment of the present invention;

FIG. 7 is a block diagram illustrating a content zone screen that may bedisplayed by the video display system of FIG. 1 in accordance with oneembodiment of the present invention;

FIG. 8 is a block diagram illustrating a content zone screen showingaugmented stories that may be displayed by the video display system ofFIG. 1 in accordance with one embodiment of the present invention;

FIG. 9 is a block diagram illustrating a remote control layout for theremote control of FIG. 1 in accordance with one embodiment of thepresent invention;

FIG. 10 is a flow diagram illustrating a method for providingpersonalized news to a consumer using the content augmentationapplication of FIGS. 1 and 2 in accordance with one embodiment of thepresent invention;

FIGS. 11A-F are screen shots illustrating complementary informationscreens that may be displayed by the video display system of FIG. 1 inaccordance with several embodiments of the present invention; and

FIG. 12 is a flow diagram illustrating a method for providingcomplementary information for a video program using the contentaugmentation application of FIGS. 1 and 2 in accordance with oneembodiment of the present invention.

FIGS. 1 through 12, discussed below, and the various embodiments used todescribe the principles of the present invention in this patent documentare by way of illustration only and should not be construed in any wayto limit the scope of the invention. In the description of the inventionthat follows, the present invention is integrated into, or is used inconnection with, a television receiver. However, this embodiment is byway of example only and should not be construed to limit the scope ofthe present invention to television receivers. Thus, those skilled inthe art will understand that the principles of the present invention maybe implemented in any suitably arranged video display system.

FIG. 1 is a block diagram illustrating a video display system 100 thatis operable to provide personalized news to a consumer in accordancewith one embodiment of the present invention. For the illustratedembodiment, the video display system 100 comprises a contentaugmentation application 102 and a television receiver 104. Although thecontent augmentation application 102 is illustrated as a separatecomponent from the television receiver 104, it will be understood thatthe content augmentation application 102 may be incorporated into thetelevision receiver 104 without departing from the scope of the presentinvention.

The content augmentation application 102 is operable to receive incomingvideo, audio, text and any other suitable signals from an externalsource, such as a cable television service provider, a local antenna, asatellite, the Internet, a DVD player, a video cassette recorder, or thelike. Thus, the incoming signals may be digital signals, analog signals,Internet protocol (IP) packets, or any other suitable type of signals.

The content augmentation application 102 is also operable to providecontent to a consumer based on the received signals. For example, thecontent augmentation application 102 is operable to transmit televisionsignals from a selected channel and/or radio signals from a selectedstation to the television receiver 104. A channel or station may beselected manually by the consumer, automatically by a recording devicepreviously programmed by the consumer, or by any other suitable means.

The content augmentation application 102 is also operable to augment thecontent received from the external sources for the consumer. Forexample, as described in more detail below, the content augmentationapplication 102 is operable to personalize news content received fromthe external sources for a consumer, to provide complementaryinformation related to the content when requested by the consumer, andto further augment the content for use by the consumer in any suitablemanner.

The content augmentation application 102 may comprise an infrared (IR)sensor 110 that is operable to receive commands from a remote controldevice 112 operated by the consumer. The content augmentationapplication 102 also comprises a user interface 114 for providingpersonalized news and complementary information to the consumer. Inorder to accomplish this, the user interface 114 comprises apersonalized news module 116 and a complementary information module 118,as described in more detail below. The television receiver 104 maycomprise a screen 120, an infrared (IR) sensor 122, and one or moremanual controls 124. The IR sensor 122 is operable to receive commandsfrom the remote control device 112 operated by the consumer.

At least a portion of the personalized news module 116 and/or thecomplementary information module 118 may comprise logic encoded inmedia. The logic comprises functional instructions for carrying outprogram tasks. The media comprises computer disks or othercomputer-readable media, application-specific integrated circuits,field-programmable gate arrays, digital signal processors, othersuitable specific or general purpose processors, transmission media orother suitable media in which logic may be encoded and utilized.Information module 118 may conform, for instance, to the teachings ofU.S. Patent Publication No. 20020144293 entitled “AUTOMATIC VIDEORETRIEVER GENIE” by N. Dimitrova et al and/or A. Janevski et al,“Enhanced Video Viewing From MetaData,” Proceedings of the SPIE ITCom,August 2001.

FIG. 2 is a block diagram illustrating the content augmentationapplication 102 in accordance with one embodiment of the presentinvention. In addition to the IR sensor 110 and the user interface 114,the content augmentation application 102 may comprise a controller 200,a processor 202, and any other suitable components (not illustrated inFIG. 2). A suitable implementation of processor 202 may be found, forexample, in A. Janevski et al, “Web Information Extraction for ContentAugmentation,” Proceedings of IEEE ICME, August 2002.

The controller 200 is operable to direct the overall operation of thecontent augmentation application 102. According to one embodiment, thecontroller 200 is operable to cause an incoming signal to be processedby the processor 202 and transmitted to the television receiver 104and/or the user interface 114. For example, the controller 200 may beoperable to cause a television signal from a cable service provider tobe demodulated and processed by the processor 202 and transmitted to thetelevision receiver 104 for viewing on the screen 120. As anotherexample, the controller 200 may be operable to cause a news story fromthe Internet to be processed by the processor 202 and transmitted to theuser interface 114 for further processing by the personal news module116.

The processor 202 may comprise radio frequency (RF) front-end circuitryfor receiving incoming television signals, tuning to a user-selectedchannel, and converting the selected RF signal to a baseband televisionsignal suitable for display on the television receiver 104. Theprocessor 202 is also operable to receive a conventional signal from anMPEG encoder/decoder and video frames from memory and to transmit abaseband television signal to the television receiver 104. In addition,the processor 202 may be operable to process video, audio, text andother suitable signals from other external sources, such as radiosignals or signals from the Internet. It will be understood that theprocessor 202 may comprise one or more processors for processing thedifferent types of signals.

The personalized news module 116 of the user interface 114 comprises amemory 210 and a processor 212. The memory 210 may comprise randomaccess memory (RAM), a combination of RAM and read only memory (ROM), orany other suitable form of memory. According to one embodiment, thememory 210 may comprise a non-volatile RAM, such as flash memory. Thememory 210 is operable to store a plurality of tables 214, 216 and 218and personalized news 219. The tables comprise user profile tables 214,profile data matching tables 216 and an information content matchingtable 218. The personalized news 219 comprises news stories and otherinformation that is identified and stored for the consumer based on theconsumer's profile. The consumer's profile may include informationrelevant to the consumer as stored in the user profile tables 214 andthe profile data matching tables 216. In addition, the memory 210 isoperable to store information identifying the format in which theconsumer desires to receive news.

The personalized news module 116 is operable to prioritize informationfor a consumer based on a personal profile of the consumer, broadcasterimportance, and other suitable criteria, as stored in the tables 214,216 and/or 218, to provide content to the consumer from multiplesources, such as television, radio, the Internet and the like, toprovide content to the consumer in multiple formats, such as video,audio and/or text, and to provide personalized news for the consumer inany other suitable manner.

In operation, according to one embodiment, information from a variety ofsources may be provided to a consumer on a single device, such as atelevision set. For example, news stories from the Internet, radiostations, and broadcast or cable television channels may be provided tothe consumer on the single device.

In addition, the consumer may choose to receive the information in avariety of different formats, such as audio, visual and/or text. Forthose stories which are from a non-visual source, such as web stories,the personalized news module 116 generates a synthetic news anchor that“reads” the story to the consumer when the consumer desires to receivethe information in a visual format. The personalized news module 116 mayalso generate a text story based on a television news story when theconsumer desires to receive the information in a text format. Similarly,the personalized news module 116 may transform stories into any formatin which the consumer desires to receive the information.

The processor 212 of the personal news module 116 identifies thosestories of interest to the consumer based on the user profile tables 214and the profile data matching tables 216. After identifying thesestories, the processor 212 stores the stories in the personalized news219 of the personalized news module 116. The processor 212 alsoprioritizes the stories based on the information content matching table218 and presents the stories to the consumer when requested in thedesired format.

In addition, for one embodiment, the personal news module 116 may storedata for a plurality of consumers in the memory 210. For thisembodiment, each consumer may provide identifying information to thepersonal news module 116 when using the user interface 114 to receivepersonalized news for that consumer.

The complementary information module 118 of the user interface 114comprises a memory 220 and a processor 222. It will be understood thatthe processors 212 and 222 may be implemented in a single processor inthe user interface 114 without departing from the scope of the presentinvention. The memory 220 may comprise random access memory (RAM), acombination of RAM and read only memory (ROM), or any other suitableform of memory. According to one embodiment, the memory 220 may comprisea non-volatile RAM, such as flash memory. The memory 210 is operable tostore user data tables 224 and complementary information 226, asdescribed in more detail below.

The complementary information module 118 is operable to provideinformation that is complementary to a video program to a consumer whenthat information is requested by the consumer. The complementaryinformation module 118 is operable to perform this task by anticipatingthe information that may be requested and storing such information inthe complementary information 226 of the memory 220, as well as storinguser data that is specific to the consumer in the user data tables 224.In an alternative embodiment, the complementary information may beretrieved by the processor 222 from the video program itself as it isreceived from an external source.

For one embodiment, the complementary information module 118 is operableto provide a visual indicator to notify the consumer that complementaryinformation is available upon request. For example, a “$” may notify theconsumer that “how much” information is available and a “w” may notifythe consumer that “who” information is available. It will be understoodthat the visual indicators may comprise any suitable form withoutdeparting from the scope of the present invention.

The data stored in the user data tables 224 may comprise dataidentifying video programs that have been previously viewed by aconsumer. This data may be stored automatically by the processor 222when a video program is played and/or may be stored manually by theconsumer. In addition, for one embodiment, the user data tables 224 maystore such data for a plurality of consumers. For this embodiment, eachconsumer may provide identifying information to the complementaryinformation module 118 when using the user interface 114 to receivecomplementary information for a video program and/or to allow thecomplementary information module 118 to store the relevant user data inthe user data tables 224 for that consumer.

The complementary information module 118 is also operable to synchronizecomplementary information 226 and a video program for the consumer, toprovide control over access and navigation of the complementaryinformation 226 to the consumer, to update the complementary information226, and to provide complementary information 226 to the consumer by anyother suitable means.

In operation, according to one embodiment, the complementary informationmodule 118 may receive queries from consumers, such as “who queries,”“what queries,” “where queries,” “when queries,” “why queries,” “howmuch queries,” and/or any other suitable queries. The complementaryinformation module 118 may then predict what information the consumerdesires and access the complementary information 226 in order toretrieve that information. If the information provided by thecomplementary information module 118 is not the information that theconsumer desired, the consumer may submit an additional query until thecorrect information is provided by the complementary information module118. In an alternative embodiment, the consumer may specify whatinformation he or she desires based on options provided by thecomplementary information module 118.

For one embodiment, the consumer may select an option that directs thecomplementary information module 118 to provide all the complementaryinformation 226 for a particular video program or to provide specifiedsubsets of the complementary information 226 for the video program.

Several options may be used in order to include complementaryinformation with a video program that may be retrieved by the processor222 and/or stored in the complementary information 226 of thecomplementary information module 118. A first option involves thecreators of the video program annotating the video program during thecreation process.

For a second option, the processor 222 may process and analyze the videoprogram to extract complementary information. For example, opticalcharacter recognition technology may be used to read the credits andlearn that a certain actor plays a character named Fred. Then, theprocessor 222 may analyze the visual, audio and transcript data forreferences to Fred and infer which onscreen character corresponds toFred. The processor 222 may then learn to recognize the face and voiceof the actor playing the part of Fred. Face recognition and voiceidentification for automatic person identification can be used for thispurpose (see: Dongge Li, Gang Wei, Ishwar K. Sethi, and NevenkaDimitrova, Person Identification in TV Shows, Journal on ElectronicImaging, special issue on Storage, Processing and Retrieval of DigitalMedia, October 2001).

A third option may combine the first and second options. For thisoption, the creators of the video program may insert any pertinentinformation they wish to include as complementary information, while theprocessor 222 may be able to analyze the video program in order toprovide additional complementary information. For example, if a consumerrequests information on an object in a scene, such as a wineglass, thatis not included by the creators, the processor 222 may attempt torecognize the object and seek information about the object from theInternet or other suitable external data sources.

The complementary information module 118 may use at least two methods inorder to update complementary information for a video program. First,the processor 222 may analyze the complementary information providedwith the video program in order to determine how much time has passedsince the complementary information was inserted or updated. If apredetermined amount of time has passed, the processor 222 may attemptto retrieve more updated information from external data sources and tostore any such retrieved information as complementary information 226for that video program.

Second, if the video program is being broadcast after a predeterminedamount of time has passed since the complementary information wasinserted or updated, the broadcasters may update the complementaryinformation before the broadcast. In addition, local broadcasters mayprovide location-specific complementary information based on cities, zipcodes, and the like for the area which will receive the broadcast.

In order to access the complementary information for a video program,consumers may use one of at least three different options. First, theconsumer may generate a query, such as a “who query,” a “what query,” a“where query,” a “when query,” a “why query,” a “how much query,” and/orany other suitable query. According to one embodiment, the “who query”may comprise one or more of filmography, biography and rumors for anactor.

The “how much” query may be provided in accordance with the principlesdescribed in the U.S. Patent Application entitled. “A System for andMethod of Shopping Through Television,” by N. Dimitrova, J. Zimmerman,and R. Jasinschi (Attorney docket no. 702359) which was filed on Dec.11, 2001.

The complementary information module 118 may then predict whatinformation the consumer desires and access the complementaryinformation in order to retrieve that information. If the informationprovided by the complementary information module 118 is not theinformation that the consumer desired, the consumer may submit anadditional query until the correct information is provided by thecomplementary information module 118.

Second, the consumer may select an option that directs the complementaryinformation module 118 to provide all the complementary information fora particular video program or to provide one or more specified subsetsof the complementary information for the video program, such as actors'names, director's comments, or the like.

Third, the consumer may request that the complementary information besent to a secondary screen, such as a personal digital assistant, atouch-screen remote control, a web pad, a mobile phone, or the like. Forthis option, the information provided to the secondary screen may begenerated in accordance with either the first or second option or by anyother suitable process.

Using the complementary information module 118, the user interface 114may allow consumers to receive a variety of types of complementaryinformation while viewing a video program. For example, a consumer maybe watching a movie and recognize an actress but be unable to rememberher name. The complementary information module 118 may provide thisinformation to the consumer, along with telling the consumer whichprevious video programs the consumer has seen with that same actressbased on the data in the user data tables 224. As another example, whileviewing a soccer game, a consumer may select a particular player andreceive statistics and other relevant information about that player fromthe complementary information module 118. Also, if a consumer is viewinga video program and sees an item that he or she would like to purchase,such as the shoes a particular character is wearing, the consumer mayreceive from the complementary information module 118 information suchas a price and/or the name of a local retail outlet at which the itemmay be purchased. For one embodiment, the scene may be bookmarked by theconsumer such that he or she may return at a more convenient time toreceive the information. For this embodiment, any complementaryinformation desired by the consumer may be identified by the consumer asdelayed complementary information. This delayed complementaryinformation may be viewed by the consumer at any later time, such as atthe end of the video program.

FIGS. 3A-F illustrate the user profile tables 214 in accordance with oneembodiment of the present invention. Although they are illustrated anddiscussed in table format, it will be understood that the user profiletables 214 may comprise any suitable data store operable to store therelevant information in any suitable manner.

FIG. 3A illustrates a personal profile table 300 that is operable tostore personal data related to the consumer. For the illustratedembodiment, the personal profile table 300 comprises a name, a homeaddress and a work address for the consumer. However, it will beunderstood that the personal profile table 300 may comprise any suitablepersonal data for the consumer without departing from the scope of thepresent invention.

FIG. 3B illustrates a traffic route table 310 that is operable to storetraffic data related to the consumer. For the illustrated embodiment,the traffic route table 310 comprises a plurality of routes from theconsumer's home to various destinations, including work. Each route(given in the left column) comprises an identification of the majorroads over which the consumer typically travels to reach thecorresponding destination (right column). It will be understood that thetraffic route table 310 may comprise any suitable traffic data for theconsumer without departing from the scope of the present invention.

FIG. 3C illustrates a traffic hot spot table 320 that is operable tostore additional traffic data related to the consumer. For theillustrated embodiment, the traffic hot spot table 320 comprises alisting of the major roads over which the consumer typically travels.However, it will be understood that the traffic hot spot table 320 maycomprise any suitable additional traffic data for the consumer withoutdeparting from the scope of the present invention. In addition, it willbe understood that the traffic route table 310 and the traffic hot spottable 320 may be consolidated into a single traffic table withoutdeparting from the scope of the present invention.

FIG. 3D illustrates a financial table 330 that is operable to storefinancial data related to the consumer. For the illustrated embodiment,the financial table 330 comprises three columns: an identifying symbol,a type indicator, such as index, stock or mutual fund, and an entityname for each of a plurality of entities in which the consumer has aninterest, financially or otherwise. However, it will be understood thatthe financial table 330 may comprise any suitable financial data for theconsumer without departing from the scope of the present invention.

FIG. 3E illustrates a sports table 340 that is operable to store sportsdata for the consumer. For the illustrated embodiment, the sports table340 comprises two columns: a sport name and a team name for each of aplurality of sports teams in which the consumer has an interest.However, it will be understood that the sports table 340 may compriseany suitable sports data without departing from the scope of the presentinvention.

FIG. 3F illustrates a headline table 350 that is operable to storeheadline data for the consumer. For the illustrated embodiment, theheadline table 350 comprises participants 352, locations 354 andkeywords 356. However, it will be understood that the headline table 350may comprise any suitable headline data for the consumer withoutdeparting from the scope of the present invention.

For the illustrated embodiment, the participants 352 comprise aplurality of potential participants in news stories in which theconsumer has an interest. For this embodiment, the participants 352 mayrefer to other user profile tables 214, such as the sports table 340(all tracked sports teams), the financial table 330 (all trackedcompanies—stock), or the like, in order to identify the participants. Inaddition, specific participants, such as China, Bush, or the like, maybe identified. The locations 354 comprise a plurality of potentiallocations from which news stories may originate or to which news storiesmay be relevant. Similarly, the keywords 356 comprise a plurality oftopics or keywords that may appear in news stories of interest to theconsumer.

FIGS. 4A-C illustrate the profile data matching tables 216 in accordancewith one embodiment of the present invention. Although they areillustrated and discussed in table format, it will be understood thatthe profile data matching tables 216 may comprise any suitable datastore operable to store the relevant information in any suitable manner.

FIG. 4A illustrates a local events table 400 that is operable to storeprioritization data for local events for the consumer. For theillustrated embodiment, the local events table 400 provides forprioritizing local events based on a match to the consumer's profile,the distance of the event from the consumer's home, and an amount oftime remaining until the event occurs. However, it will be understoodthat the local events table 400 may provide for prioritizing localevents based on any suitable criteria without departing from the scopeof the present invention. According to one embodiment, the consumer mayspecify the percentages associated with each criterion in the localevents table 400. In addition, according to one embodiment, the criteriathemselves may be selected by the consumer. Alternatively, thepercentages and/or the criteria may be predetermined.

FIG. 4B illustrates a headline table 410 that is operable to storeprioritization data for headlines for the consumer. For the illustratedembodiment, the headline table 410 provides for prioritizing headlinesbased on a match to the consumer's profile, broadcaster importance, asdescribed in more detail below in connection with FIG. 5, and howcurrent the headline is. However, it will be understood that theheadline table 410 may provide for prioritizing headlines based on anysuitable criteria without departing from the scope of the presentinvention. According to one embodiment, the consumer may specify thepercentages associated with each criterion in the headline table 410. Inaddition, according to one embodiment, the criteria themselves may beselected by the consumer. Alternatively, the percentages and/or thecriteria may be predetermined.

FIG. 4C illustrates a story table 420 that is operable to storeprioritization data for stories other than headlines for the consumer.For the illustrated embodiment, the story table 420 provides forprioritizing stories other than headlines based on a match to theconsumer's profile, broadcaster importance, as described in more detailbelow in connection with FIG. 5, and an amount of time since or untilthe occurrence of an event associated with the story. However, it willbe understood that the story table 420 may provide for prioritizingstories based on any suitable criteria without departing from the scopeof the present invention. According to one embodiment, the consumer mayspecify the percentages associated with each criterion in the storytable 420. In addition, according to one embodiment, the criteriathemselves may be selected by the consumer. Alternatively, thepercentages and/or the criteria may be predetermined.

FIG. 5 illustrates the information content matching table 218 inaccordance with one embodiment of the present invention. Although theyare illustrated and discussed in table format, it will be understoodthat the information content matching table 218 may comprise anysuitable data store operable to store the relevant information in anysuitable manner.

The information content matching table 218 is operable to store datarelated to prioritization of news for the consumer. For the illustratedembodiment, the information content matching table 218 comprisesbroadcaster importance, story depth, and novelty. However, it will beunderstood that the information content matching table 218 may compriseany suitable prioritization data without departing from the scope of thepresent invention.

Broadcaster importance is a relative measure of the importance of a newsstory based on length, position within a broadcast or on a web page, orother suitable criteria. Story depth is an absolute measure of theimportance of a news story based on length, new information ofsubstance, quality of the reportage, such as who is interviewed andwhether there is relevant action in the reportage, or other suitablecriteria Novelty is a measure of the overall novelty of the informationwithin a news story based whether the story relates to a new event, isan update of an earlier story or is a continuing story about an earlierevent.

According to the illustrated embodiment, the relative importance ofbroadcaster importance and novelty combine to 100%, while the storydepth may comprise a binary filter of shallow or deep. However, it willbe understood that these criteria may be otherwise weighted (forexample, all of them can be binary filters or can have explicit numericvalues) without departing from the scope of the present invention.

For one embodiment, the broadcaster importance may comprise a match torank ordered stories and an amplification-based weighted measure, withthe top stories receiving the heaviest matching. The story depth maycomprise a match to select which story to present based on the binaryfilter. With clustered stories, only one story may be presented, withthe consumer able to request the additional stories. The novelty maycomprise a match to select the novel stories and follow up on ongoingstories.

FIG. 6 is a block diagram illustrating a summary screen 600 that may bedisplayed by the video display system 100 in accordance with a firstembodiment of the present invention. The summary screen 600 comprises abanner 602 and a plurality of content zone summaries 604, 606, 608, 610,612 and 614. The banner 602 may indicate the logo of the serviceprovider, the number of stories found in each of the content zones, thetotal duration of those stories, and a current date and time.

Although the illustrated embodiment comprises six content zone summaries604, 606, 608, 610, 612 and 614 with six corresponding content zones, itwill be understood that any suitable number of any suitable type ofcontent zone may be provided without departing from the scope of thepresent invention. The content zone summaries 604, 606, 608, 610, 612and 614 comprise summaries of the information available in each of thecorresponding content zones. For the illustrated embodiment, the contentzone summaries comprise a weather zone 604, a traffic zone 606, afinancial zone 608, a headline zone 610, a local event zone 612, and asports zone 614.

While viewing the summary screen 600, a consumer may select one of thecontent zones in order to view a content zone screen with additionaldetails and stories, direct the content augmentation application 102 toplay all news stories in a particular content zone, direct the contentaugmentation application 102 to play all news stories in all contentzones, direct the content augmentation application 102 to play selectedstories from all content zones and/or select any other suitable task forthe content augmentation application 102 to perform.

FIG. 7 is a block diagram illustrating a content zone screen 700 thatmay be displayed by the video display system 100 in accordance with oneembodiment of the present invention. For this embodiment, the contentaugmentation application 102 begins to play news stories when thecontent zone screen 700 is selected for display by the consumer. Basedon the consumer's profile, the content augmentation application 102places the content zones in a particular order for playback.

The content zone that comprises the news stories currently being played,or the expanded content zone, is identified by a content zone label 702,and the stories for that content zone are played in an expanded zone704. Alternatively, the content zone may be identified by a specificbackground pattern, with specific texture and/or animation. A channellabel 706 identifies the source of the news story currently being playedin the expanded zone 704, and the placement of the news story withinthat source is identified by a placement bar 708. The duration of thenews story, as well as the elapsed time, is provided in a duration bar710. An additional information section 712 may also identify theposition of the story relative to the total number of stories for theexpanded content zone, the status of the currently playing story, suchas selected, new, seen, partially seen, saved, and the like, and anyother relevant information.

A collapsed content zone section 720 comprises an identifier, a summary,or other suitable information corresponding to each of the content zonesother than the expanded content zone. These collapsed content zones maybe listed in the order in which they will be selected for playback basedon the consumer's profile. Thus, as the news stories for each expandedcontent zone are completed, the expanded content zone may be moved tothe bottom of the collapsed content zone section 720 and the contentzone at the top of the collapsed content zone section 720 may become thenext expanded content zone.

At any time during the playback of a story, the consumer may also selecta full screen option, which causes the story to be played using the fullscreen 120. Thus, the collapsed content zone section 720, as well as thechannel label 706, the placement bar 708, the duration bar 710, and theadditional information section 712, may be hidden, allowing the expandedzone 704 to fill the entire screen 120. It will be understood that thefull screen option may be modified to include allow the display of anyof the channel label 706, the placement bar 708, the duration bar 710,and the additional information section 712 along with the expanded zone704 without departing from the scope of the present invention.

FIG. 8 is a block diagram illustrating a content zone screen 800 thatmay be displayed by the video display system 100 in accordance with oneembodiment of the present invention. A consumer may select a particularcontent zone from a summary screen, such as the summary screen 600 or700. The selected content zone is identified by a content zone label802. Alternatively, the content zone may be identified by a specificbackground pattern, with specific texture and/or animation.

If a consumer directs the content augmentation application 102 to playall the news stories for the content zone, the stories for that contentzone are played in an expanded zone 804. A channel label 806 identifiesthe source of the news story currently being played in the expanded zone804, and the placement of the news story within that source isidentified by a placement bar 808. The duration of the news story, aswell as the elapsed time, is provided in a duration bar 810. Anadditional information section 812 may also identify the position of thestory relative to the total number of stories for the selected contentzone, the status of the currently playing story, such as selected, new,seen, partially seen, saved, and the like, and any other relevantinformation.

A collapsed story section 820 comprises an identifier, a summary, orother suitable information corresponding to each of the stories otherthan the currently playing story. These collapsed stories may beprioritized in accordance with the data stored in the informationcontent matching table 218, as described in more detail above inconnection with FIG. 5. Thus, as each news story is completed, thatstory may be moved to the bottom of the collapsed story section 820 andthe story at the top of the collapsed story section 820 may become thenext expanded story to be played in the expanded zone 804. In addition,the consumer may manually select any one of the stories from thecollapsed story section 820 for playback at any time.

At any time during the playback of a story, the consumer may also selecta full screen option, which causes the story to be played using the fullscreen 120. Thus, the collapsed story section 820, as well as thechannel label 806, the placement bar 808, the duration bar 810, and theadditional information section 812, may be hidden, allowing the expandedzone 804 to fill the entire screen 120. It will be understood that thefull screen option may be modified to include allow the display of anyof the channel label 806, the placement bar 808, the duration bar 810,and the additional information section 812 along with the expanded zone804 without departing from the scope of the present invention.

The content zone screen 800 comprises a clustered story button 824 thatis operable to identify a story that has been augmented with additionalinformation from the Internet, from other television channels, or thelike, after the original story was posted or broadcast. By selecting astory identified by a clustered story button 824, the consumer may seethis additional information. It will be understood that the position ofthe visual elements in the content zone screen 800 may be flexible andmay be configurable by the consumer.

FIG. 9 is a block diagram illustrating a remote control layout 900 forthe remote control 112 in accordance with one embodiment of the presentinvention. For this embodiment, the remote control 112 may allow aconsumer to operate the personal news module 116 of the contentaugmentation application 102. The layout 900 may comprise two screens: azones screen 902 and an action screen 904. It will be understood thatthe layout 900 may comprise a single screen or more than two screenswithout departing from the scope of the present invention.

The zones screen 902 comprises an action button 906, a weather button908, a traffic button 910, a financial button 912, a sports button 914,an events button 916, and a headlines button 918. The action button 906transitions to the summary screen 600 or 700 and transitions to theaction screen 904. The weather button 908, the traffic button 910, thefinancial button 912, the sports button 914, the events button 916, andthe headlines button 918 transition to the corresponding content zonescreen 800.

The action screen 904 comprises a zones button 920, an info button 922,a play/pause button 924, a previous button 926, a next button 928, aplay all button 930, a screen button 932, a profiles button 934, and areset button 936. The zones button 920 transitions to the zones screen902. The info button 922 extracts or contracts clustered stories for aselected story on a content zone screen 800. The play/pause button 924toggles the story being played between play and pause.

The previous button 926 skips to the previous story. The next button 928skips to the next story. When the previous button 926 or the next button928 is pressed by the consumer, the previous or next story,respectively, starts playing automatically. The play all button 930plays all news stories for a content zone or, from the summary screen600, for all content zones. Thus, all news stories are strung togetherto form a personalized television show. For one embodiment, all the newsstories may be played back from the summary screen 600 based on absolutepriority regardless of the content zones.

The screen button 932 toggles between a full screen option and a summaryor content zone screen 600, 700 or 800. The profiles button 934 togglesbetween a traffic profile and the traffic content zone. The reset button936 returns all values to default values and starts a demonstration fromthe beginning.

FIG. 10 is a flow diagram illustrating a method for providingpersonalized news to a consumer using the content augmentationapplication 102 in accordance with one embodiment of the presentinvention. The method begins at step 1000 where the content augmentationapplication 102 receives news from external sources, such as a cabletelevision service provider, a local antenna, a satellite, the Internet,a DVD player, a video cassette recorder, or the like. The news maycomprise news stories and other newsworthy information, such as weatherand traffic related information.

At step 1002, the processor 212 of the personal news module 116identifies a subset of the received news for the consumer based on theconsumer's profile. At step 1004, the processor 212 stores theidentified news in the personalized news 219.

At step 1006, the content augmentation application 102 receives anactivation request from the consumer. For example, the consumer maypress a button on the remote control 112 in order to send the activationrequest to the infrared sensor 110 of the content augmentationapplication 102. The activation request comprises a request for thecontent augmentation application 102 to provide the news stored in thepersonalized news 219 to the consumer.

At step 1008, the processor 212 prioritizes the news stored in thepersonalized news 219 based on data stored in the information contentmatching table 218. At step 1010, the personal news module 116 providesa summary screen 600 or 700 to the consumer.

At decisional step 1012, a determination is made regarding whether ornot the consumer has requested to view a particular content zone. If theconsumer has not requested to view a particular content zone, the methodfollows the No branch from decisional step 1012 and returns to step 1000where the content augmentation application 102 continues to receive newsfrom external sources. Thus, the content augmentation application 102may receive news twenty-four hours a day, seven days a week. However, ifthe consumer has requested to view a particular content zone, the methodfollows the Yes branch from decisional step 1012 to step 1014. At step1014, the personal news module 116 provides the requested content zonescreen 800 to the consumer.

At decisional step 1016, a determination is made regarding whether ornot the consumer has requested to view another content zone. If theconsumer has requested to view another content zone, the method followsthe Yes branch from decisional step 1016 and returns to step 1014, wherethe personal news module 116 provides the requested content zone screen800 to the consumer. However, if the consumer has not requested to viewanother content zone, the method follows the No branch from decisionalstep 1016 to decisional step 1018.

At decisional step 1018, a determination is made regarding whether ornot the consumer has requested to view the summary screen 600 or 700. Ifthe consumer has requested to view the summary screen 600 or 700, themethod follows the Yes branch from decisional step 1018 and returns tostep 1010, where the personal news module 116 provides the summaryscreen 600 or 700 to the consumer. However, if the consumer has notrequested to view the summary screen 600 or 700, the method follows theNo branch from decisional step 1018 and returns to step 1000 where thecontent augmentation application 102 continues to receive news fromexternal sources.

FIGS. 11A-F are screen shots illustrating complementary informationscreens 1100 a-f that may be displayed by the video display system 100in accordance with several embodiments of the present invention. FIG.11A comprises a complementary information screen 1100 a that illustratesone embodiment of a complementary information display 1102 a that may bedisplayed by the complementary information module 118 in response to a“what query” from a consumer. In this example, the complementaryinformation module 118 predicts that the consumer is requestinginformation on the gun. As described in more detail above in connectionwith FIG. 2, the consumer may initiate a subsequent “what query” if thegun is not the object about which the consumer desires more information.

FIG. 11B comprises a complementary information screen 1100 b thatillustrates a first embodiment of a complementary information display1102 b that may be displayed by the complementary information module 118in response to a “who query” from a consumer. In this example, thecomplementary information module 118 predicts that the consumer isrequesting information about the actress in the foreground. As describedin more detail above in connection with FIG. 2, the consumer mayinitiate a subsequent “who query” if the consumer desired informationabout the actor in the background instead.

FIG. 11C comprises a complementary information screen 1100 c thatillustrates a second embodiment of a complementary information display1102 c that may be displayed by the complementary information module 118in response to a “who query” from a consumer. In this example, thecomplementary information module 118 predicts that the consumer isrequesting information about the actress. As described in more detailabove in connection with FIG. 2, the consumer may initiate a subsequent“who query” if the consumer desired information about the actor instead.

FIG. 11D comprises a complementary information screen 1100 d thatillustrates a third embodiment of a complementary information display1102 d that may be displayed by the complementary information module 118in response to a “who query” from a consumer. In this example, thecomplementary information module 118 allows the consumer to select aparticular actor from a plurality of images 1104. In addition, for thisembodiment, the consumer may select from at least two informationoptions 1106, such as “movies” and “rumors” and the like, to view aparticular type of information related to the selected actor.

FIG. 11E comprises a complementary information screen 1100 e thatillustrates one embodiment of detailed complementary information thatmay be displayed by the complementary information module 118. In thisembodiment, the complementary information module 118 allows the consumerto select from at least two query options 1108, such as “music,” “cast,”“location” and the like, to view additional information and/or optionsfor selection.

For example, in the illustrated embodiment, the consumer has firstselected the “cast” query option 1108, causing the complementaryinformation module 118 to provide a plurality of images 1110 of castmembers from which the consumer may select. The consumer has thenselected the first cast member, Shirley Appleby, causing thecomplementary information module 118 to provide information aboutShirley Appleby to the consumer in a complementary information display1102 e. Also, for the illustrated embodiment, the complementaryinformation display 1102 e indicates to the consumer by highlightingthat the consumer has seen Shirley Appleby playing the role of BridgetManilla in the movie, “The Thirteenth Floor.” This information may beretrieved by the complementary information module 118 from the user datatables 224.

FIG. 11F comprises a complementary information screen 1100 f thatillustrates another embodiment of detailed complementary informationthat may be displayed by the complementary information module 118. Inthis embodiment, the complementary information module 118 allows theconsumer to select from at least two query options 1112, such as“location,” “clothes,” “music,” “cast,” and the like, to view additionalinformation and/or options for selection.

For example, in the illustrated embodiment, the consumer has firstselected the “music” query option 1112, causing the complementaryinformation module 118 to provide a plurality of images 1114 ofrecordings containing songs played in the corresponding video programfrom which the consumer may select. The consumer has then selected thefirst song, Dido's “Here With Me,” causing the complementary informationmodule 118 to provide information about the recording “No Angel,” onwhich the song “Here With Me” appears, in a complementary informationdisplay 1102 f. It will be understood that the position of the visualelements in the complementary information screens 1100 may be flexibleand may be configurable by the consumer.

FIG. 12 is a flow diagram illustrating a method for providingcomplementary information for a video program using the contentaugmentation application 102 in accordance with one embodiment of thepresent invention. The method begins at step 1200 where thecomplementary information module 118 receives complementary informationregarding a video program. The complementary information may be receiveddirectly from the video program and/or retrieved from the complementaryinformation 226 in the memory 220 of the complementary informationmodule 118.

At decisional step 1202, a determination is made regarding whether ornot the complementary information needs to be updated. According to oneembodiment, this determination is made based on whether or not apredetermined amount of time has passed since the complementaryinformation was inserted or updated in the video program. If thecomplementary information needs to be updated, the method follows theYes branch from decisional step 1202 to step 1204. At step 1204, thecomplementary information module 118 attempts to retrieve updatedinformation from external data sources and to store any such retrievedinformation as complementary information 226 for that video program.

Returning to decisional step 1202, if the complementary information doesnot need to be updated, the method follows the No branch from decisionalstep 1202 to step 1206. Also, from step 1204, the method continues tostep 1206. At step 1206, the complementary information module 118receives a query from a consumer. The query may comprise a “what query,”a “who query,” a “music query,” or any other suitable type of queryoperable to indicate to the complementary information module 118 whattype of information the consumer desires to receive. The music query canbe a query by humming, by matching the musical phrase in the movie toall known songs, or by any other suitable method.

At step 1208, the complementary information module 118 provides a queryresponse to the consumer. In providing this response, the complementaryinformation module 118 may access the user data tables 224 in order toinclude any consumer-specific data relevant to the response. Forexample, if the query is a “who query,” the complementary informationmodule 118 may access the user data tables 224 in order to determinewhat, if any, video programs the consumer has previously seen with thesame actor that is identified by the complementary information module118 in the query response.

At decisional step 1210, a determination is made regarding whether ornot an additional query has been received from the consumer. If anadditional query has been received from the consumer, the method followsthe Yes branch from decisional step 1210 and returns to step 1208, wherethe complementary information module 118 provides a query response tothe consumer. However, if no additional query has been received from theconsumer, the method follows the No branch from decisional step 1210 andcomes to an end.

Although the present invention has been described with severalembodiments, various changes and modifications may be suggested to oneskilled in the art. It is intended that the present invention encompasssuch changes and modifications as fall within the scope of the appendedclaims.

1. A method for providing complementary information for a video program,comprising: receiving complementary information for a video program;receiving a query from a consumer, the query related to a specifiedportion of the complementary information; and providing a query responseto the consumer based on the specified portion of the complementaryinformation.
 2. The method of claim 1, further comprising: determiningwhether a predetermined amount of time has passed since thecomplementary information was updated in the video program; andretrieving updated information from at least one external data sourcewhen the predetermined amount of time has passed since the complementaryinformation was updated.
 3. The method of claim 1, receivingcomplementary information comprising receiving complementary informationdirectly from the video program.
 4. The method of claim 1, furthercomprising anticipating the specified portion of the complementaryinformation and storing the anticipated information.
 5. The method ofclaim 1, providing a query response comprising accessing user datatables for consumer-specific data and personalizing the query responsebased on the consumer-specific data.
 6. The method of claim 5, theconsumer-specific data comprising data identifying video programspreviously viewed by the consumer.
 7. The method of claim 6, furthercomprising automatically storing the consumer-specific data identifyingvideo programs previously viewed by the consumer when the video programsare viewed by the consumer.
 8. The method of claim 6, further comprisingstoring the consumer-specific data identifying video programs previouslyviewed by the consumer based on manual input by the consumer.
 9. Themethod of claim 1, further comprising extracting complementaryinformation from the video program using automatic personidentification.
 10. The method of claim 1, further comprising receivinga request that the complementary information be sent to a secondaryscreen.
 11. The method of claim 1, visual elements of the query responseconfigurable by the consumer.
 12. The method of claim 1, furthercomprising receiving from the consumer an identification ofcomplementary information as delayed complementary information andproviding the delayed complementary information to the consumer at alater time.
 13. The method of claim 1, the query comprising one of a whoquery, a what query, a where query, a when query, a why query and a howmuch query.
 14. The method of claim 1, further comprising providing avisual indicator to notify the consumer that complementary informationis available upon request.
 15. The method of claim 1, further comprisingdisplaying the complementary information on a secondary screensynchronously with the video program.
 16. A system for providingcomplementary information for a video program, comprising: a memoryoperable to store user data tables and complementary information; and aprocessor operable to receive complementary information for a videoprogram, to receive a query from a consumer, the query related to aspecified portion of the complementary information, and to provide aquery response to the consumer based on the specified portion of thecomplementary information.
 17. The system of claim 16, the processorfurther operable to determine whether a predetermined amount of time haspassed since the complementary information was updated in the videoprogram and to retrieve updated information from at least one externaldata source when the predetermined amount of time has passed since thecomplementary information was updated.
 18. A system for providingcomplementary information for a video program, comprising: acomputer-processable medium; and logic stored on thecomputer-processable medium, the logic operable to receive complementaryinformation for a video program, to receive a query from a consumer, thequery related to a specified portion of the complementary information,and to provide a query response to the consumer based on the specifiedportion of the complementary information.